Summary

• Most benchmarks call sync && echo 3 > /proc/sys/vm/drop_caches at the start of run.sh to prepare for a cold first run.
• That sync also flushes any dirty pages of the source dataset files (hits.tsv, hits.csv, hits.parquet, hits.json.gz, etc.) that were downloaded and loaded into the DB but are no longer needed.
• Cost of that incidental flush depends on what input format the system happened to ingest (~70 GB uncompressed TSV/CSV vs ~14 GB Parquet, plus per-system layout differences), so it leaks into cold-run measurements unevenly across systems — effectively a hidden violation of benchmark rules.
• Fix: in benchmark.sh of every system that ingests the dataset into its own storage format, delete the downloaded source files immediately after the load step (and after any data_size measurement that depends on them) and before ./run.sh is invoked. The unlinked files' dirty pages are dropped by the kernel without being flushed, so the subsequent sync covers only the database's own writes.
• Systems that query the parquet/csv/tsv file directly as their storage (clickhouse-parquet, duckdb-parquet, datafusion, polars, sail, etc.) are intentionally not modified — those files ARE the data and must remain.
• Skipped: locustdb (script panics during load and never reaches run.sh), mongodb (uses run.js, not run.sh).

33 systems modified: byconity, cedardb, chdb, citus, clickhouse, cloudberry, cockroachdb, cratedb, databend, doris, druid, duckdb, duckdb-vortex, elasticsearch, greenplum, heavyai, hologres, hyper, infobright, kinetica, mariadb, mariadb-columnstore, monetdb, mysql, mysql-myisam, oxla, pg_duckdb, pg_duckdb-indexed, pgpro_tam, pinot, selectdb, starrocks, umbra.

Test plan

• Re-run one ingest-from-TSV benchmark (e.g. starrocks, mysql) on AWS and confirm the script still completes end-to-end with the source file deleted before ./run.sh.
• Re-run one ingest-from-Parquet benchmark (e.g. duckdb, doris, clickhouse) and confirm the same.
• Spot-check that no removed file is needed by ./run.sh (run.sh only references the database state, not the source files, in modified systems).
• Verify data_size output is unchanged for systems where it's measured after ./run.sh — those measurements read internal storage, not the deleted source.

🤖 Generated with Claude Code